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1. Introduction. In this paper, we study an infinite dimensional stochastic op- 
timal problem whose purpose is to minimize a cost functional 



with the control variable u valued in a subset U of a metric space. Here the state 
variable x takes values in a separable Hilbert space H, A and B are both stochastic 
evolution operators, / and g are given H-valned functions, I and h are nonlinear 
functional, and W is a 1-dimensional standard Wiener process0. 

The research on the Pontryagin maximum principle for infinite-dimensional non- 
linear stochastic evolution systems has developed for a long time. All existing pub- 
lications considered only the case that the control does not appear in the diffusion 
term (see [Q El 02 El Q31 QH [18] etc.). Very recently, there are three preprints [7|I51H2] 
concerning the case that the control appears in the diffusion term. Both the works 
p~2l [8] discuss a simple form of system (jl.l[) with A being an infinitesimal generator 
of a strongly continuous semigroup and B = 0. In our previous work [7], a maximum 
principle is proved under the same setting as in the present paper, but the characteri- 
zation of the second-order adjoint process was indirect. In this paper, we give a direct 
and clear characterization of the 2nd-order adjoint process and establish a complete 
formulation of stochastic maximum principle for the optimal control in the general 
case that stochastic evolution operators, as well as the control variable with values in 
a general set, enter into both drift and diffusion terms of the state equation. 

A basic idea of establishing the general maximum principle for stochastic optimal 
control follows from a well-known work [Tl] which solved the finite-dimensional case. 
The main difficulties in our problem are (i) the L p -estimate for SEEs, and (ii) the 
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subject to the semilinear stochastic evolution equation (SEE) 



dx(t) = [A(t)x(t) + f(t, x(t),u(t))]dt + [B(t)x(t) + g(t, x(t),u(t))] dW t 
x(0) = x 



i • 



(1.1) 



second-order duality analysis (in other words, the characterization of the second-order 
adjoint process). To get over the former, we first introduce a structural condition 
on the operator B (Assumption [2~3"]) . which is refined from many applications. For 
the second, a classical method is to characterize the second-order adjoint process 
by an operator-valued BSDEs. This, working well for the finite-dimensional case, 
meets a big problem for the infinite-dimensional case, because the required solvability 
result of infinite-dimensional operator-valued BSDEs is unknown so far. Our approach 
of overcoming the second difficulty is novel. By utilizing the Lebesgue differential 
theorem at an early step of the second-order duality analysis (Lemma l5.4p , we simplify 
the problem into characterizing the limit of a class of bilinear functionals on H- valued 
random variables (for more explanations, see Remark I5.5p . We show that this limit 
is associated with a stochastic bilinear functional (Proposition 14.11) which can be 
represented by an operator- valued stochastic process (Theorem 12. 6p . The last thing 
is exactly the desired second-order adjoint process. 

The rest of this paper is organized as follows. In Section 2, we state the problem 
and our main results. In Section 3, the basic L p -estimate for SEEs is derived. In Sec- 
tion 4, we study the representation and properties of a stochastic bilinear functional, 
and finally in Section 5, we derive the stochastic maximum principle. 

2. Formulation and main results. 

2.1. Notations and statement of the problem. Let (f2, J 7 , F, P) be a filtered 
probability space where the filtration F = (J r t)t>o is generated by a 1-dimensional 
standard Wiener process W = {W t ;t > 0} and satisfies the usual conditions. 

Let H and V be two separable real Hilbert spaces such that V is densely embedded 
in H. We identify H with its dual space, and denote by V* the dual of V. Then we 
have V C H C V*. Denote by \\-\\ H the norms of H, by (•,•) the inner product in 
H, and by (•, •)„ the duality product between V and V*. Denote by 58(X — > Y) the 
Banach space of all bounded linear operators from Banach space X to Banach space Y, 
with the norm ||T|| := sup{||Ta:|| y ; \\x\\ x = 1}. Simply, denote 58 (X) = 58(X -> X). 
For a er-algebra Q C J 7 , denote by L^,(Q, G, %$(H)) the space of all essentially 58(i?)- 
valued weakly ^-measurable random variables P satisfying E||P|||^^ < oo. 

Now recall the controlled stochastic evolution system 

dx(t) = [A(t)x(t) + f(t, x(t),u(t))]dt + [B(t)x{t) + g(t, x(t),u(t))] dW t , 
x(0) = x 

with the control process u(-) valued in a set U, given stochastic evolution operators 

A : [0,1] x fi -> 58(V -> V*) and B : [0, 1] x Q -> 58(7 -> H), 
and nonlinear terms 

f,g:[0,l]xHxUxQ^H. 

Here the control set U is a nonempty Borel-measurable subset of a metric space whose 
metric is denoted by dist(-, •). Fix an element (denoted by 0) in U, and then define 
\u\u = dist(it, 0). An admissible control u(-) is a /7-valued F-progressively measurable 
process such that 

sup E |u(t)|^ < oo. 
te[o,i] 
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Denote by U a & the set of all admissible controls. 

Our optimal control problem is to find it(-) S £7 ac j minimizing the cost functional 

J(u(-))=E / l{t,x(t),u(t))dt + m(x(l)) 
Jo 

with given functions 

I : [0, 1] x 77 x 17 x n -> K and ft : 77 x Q ->• M. 

We make the following assumptions. Fix some constants k G (0, 1) and K G 
(0,oo). 

Assumption 2.1. TTie operator processes A and B are weakly ¥ -progressively 
measurable, i.e., (XjAtyy)^ and (x,B(t)y) are both ¥ -progressively measurable pro- 
cesses for any x, y € V ; and for each (i, us) 6 [0, 1] x CI, 

{x,A(t)x) m + \\B(t)x\\ 2 H < -k\\x\\1+K \\x\\ 2 h , 
\\A(t)x\\ 2 v , <X"||af||v • Vie eV. 

Assumption 2.2. For each (x, u) e 77 x U, /(•, x, u), g(-, x, u) and l(-,x, u) are 
all ¥ -progressively measurable processes, h(x) is J- \ -measurable random variable; for 
each (t, u, ui) € [0, 1] x U x fl, /, g, I and h are globally twice Frechet differ entiable with 
respect to x. The functions f x ,dxifxxidxx,lxx,h xx are bounded by the constant K; 
/, g, l x , h x are bounded by K{1 + \\x\\h + \ u \u); I, h is bounded by K(l + 11x11^+ 
Herein "bounded" is in the sense of their corresponding norms. 

From a well-known result (see e.g. [TU1 Theorem 2.2.1]), SEE ([l.ip has a unique F- 
progressively measurable weak solution x(-) £ C([0, 1], L 2 (fl, 77)) for each u(-) € C/ a d 
under Assumptions 12.11 and 12.21 

To establish the L p -estimate (j> > 2) for the solutions to stochastic evolution 
equations, the following structural condition on the operator B is in force. 

Assumption 2.3. For each (t,u>), 

\{x,B(t)x)\<K\\xf H , VxeV. 

Here we call this condition the quasi-skew-symmetry. 

To be more specific about this condition, we give a lemma as below. 
Lemma 2.4. Give B £ %{V —¥ 77). The following conditions are equivalent: 

(1) foranyx£V,\(x,Bx)\<K\\x\\ 2 H ; 

(2) B + B* e 95 (if), where B* is the dual operator of B; 

(3) there are a skew- symmetric operator S G 03 (V^ — >■ 77) and a symmetric opera- 
tor T E 93(if ) such that B = S + T . 

Proof. (l)-o-(2): Noticing the fact that B + B* is symmetric, this assertion follows 
from the equality 

(x, (B + B*)x) = 2{x,Bx), VxeV. 

(2)<S=>(3): The assertion from (2) to (3) follows immediately by taking S = (B — 
B*)/2 and T = (B + B*)/2. The other direction is obvious. □ 

This structural condition is refined from many applications. As an important ex- 
ample, the operator B appearing in the system governed by stochastic PDEs is usually 
a lst-order differential operator which naturally satisfies the quasi- skew- symmetric 
condition. For more aspects about SPDEs, we refer to [HE3, etc. 
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2.2. Main results. Let operator processes A(-) and B(-) satisfy Assumptions 
12.11 and 12. 31 M be a <8(-ff)-valued weakly ^-measurable random variable and N(-) 
be a 25(.ff)-valued weakly F-progressively measurable stochastic process. 

Definition 2.5. //, in addition to the above setting, 

\\M\\l { H) + f \\N(t)\\l (H) to =: Ai e L l (Q,^,R), (2.1) 
Jo 

then we say the four-tuple (A,B,M,N) is "appropriate". 

For each r £ [0,1), we formally define a stochastic bilinear functional on the 
Banach space L 4 (£l, T T , H) , associated with the four-tuple (A, B, M, N), as the form: 

[T T (A, B, M, NM, C) := (z T, ^(l),M^(l)) + J (z^(t), N(t)z^(t)) dt, 

(2.2) 

with the function z T, ^(-) (similarly for z r '^(-)) solving the equation 

z T 'Ht) = £ + J A(s)z r >t(s) ds + J B(s)z T ^(s) dW s , t £ [t, 1]. (2.3) 

Then we have the following representation theorem for this bilinear functional. 
Theorem 2.6. Let the four-tuple (A, B , M , N) be "appropriate". Then 

a) for each t £ [0,1], the stochastic bilinear functional T T (A, B, M, N) is well- 
defined, more specifically, for any £, £ £ L A (Vl, T T , H), [T T (A, B, M, N)](£, £) is uniquely^ 
determined (a.s.) and 

\[T T (A,B,M,N)] (60| < C^JQVE^A! U\\ h \\(\\ h (a.s.). (2.4) 

b) (representation) there exists a ( B(H)-valued weakly ¥ -progressively measurable 
process P. with 

\\Pt\\1 {H ) < C(k,K)E^A 1 (a.s.), We [0,1) (2.5) 

such that for each r £ [0, 1] and any £, £ £ L (ft, T T , H), 

(i,P T Q = [T T (A,B,M,N)](t,t) (a.s.). (2.6) 

We call P. the Riesz representation of T.(A,B,M,N). Hereafter C(-) is a positive 
constant depending only on the values in the brackets. 

c) (weak stochastic continuity) for each r £ [0,1) and any £, C € L A (Q,, T T , H), 
we have 

lim E (£, (P s - P t )C) = 0, with s,t £ [r, 1]. (2.7) 

|s-t|->0 

d) (uniqueness) if P} and P 2 are both the Riesz representations ofT.(A, B, M, N), 
then P^ = P 2 a.s. for each r £ [0, 1]. 

The proof of this theorem is placed in Subsection 4.1. 

Now we turn to the control problem. Define the Hamiltonian function 

H : [0, 1] x H x U x H x H ->■ K, 
1 



as the form 

H(t, x, u,p, q) := l(t, x, u) + (p, f(t, x, u)) + (q, g{t, x, u)) , (2.8) 

then our main result can be stated as follows. 

Theorem 2.7 (Stochastic Maximum Principle). Let Assumvtions \2.1\WU\ be 
satisfied. Suppose x is the state process with respect to an optimal control u. Then 

i) (first-order adjoint process) the backward stochastic evolution equation (BSEE) 

dp(t) = -[A*(t)p(t) + B*(t)q(t) + H x (t, x(t), u(t),p(t), q(t))] dt + q{i) dW t , 
p(l) = h x (x(l)). (2.9) 

has a unique ¥ -progressively measurable (weak) solution^ (p,q); 

ii) (second-order adjoint process) the four-tuple (A, B, M , N) with 

A(t) := A{t) + f x (t, x(t), u(t)), B(t) := B(t) + g x (t, x(t), u(t)), 
M := h xx (x(l)), N(t) := H x (t,x(t),u(t),p(t),q(t)) 

is "appropriate", consequently from Theorem \2. 61 there is a unique *&(H) -valued pro- 
cess P. as the Riesz representation of T.(A, B, M, N); 

iii) (maximum condition) for each u € U , the inequality 

H(t, x(t),u,p(t), q(r)) - H(t, x(t), u(t),p(t), q(r)) 

+ ~ (g(r, x(t),u) - g{T, x(t), u{t)),P t [g(t, x(t), u) - g(r, x(r), u(r))]) > (2.10) 

holds for a.e. (t,uj) € [0, 1) x Q. 

The proof of this theorem will be completed in Section 5. 

3. i p -estimates for stochastic evolution equations. The L p -estimate of the 
solutions to evolution equations plays a basic role in our approaches. Now we consider 
the following linear equation 

dy(t) = [A(t)y(t) + a(t)} dt + [B(t)y(t) + b(t)]dW t , 
2/(0) =y a eH. (3.1) 

Under Assumption 12 . 1 1 the above equation has a unique (F-progressively measurable) 
solution y(-) G C([0, T], L 2 {VL, H)) providing two H- valued F-progressively measurable 
processes a and b such that 

J 1 (E \\a(s)f H ) 5 ds +J E \\b{s)\\ 2 H ds < oo. 

If additionally the operator B satisfies the quasi- skew- symmetric condition, then we 
have 

Lemma 3.1. Let A and B satisfy Assumvtions [Ql and \2.,'A and n £ Z + . // 



a:= J (E||a(.s)||^) 2 " ds < oo, 
(3:=£ (E\\b(s)\\fy ds<oo, 



2 For the definition of (weak) solutions to BSEEs, we refer to [6] Def. 3.1] 
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then the solution to equation (|3.1[) satisfies 

sup B\\y(t)\\f < C(K,K,n) (\\y \\% + o? n + $A . 
te[o,i] ' v J 

Proof. The assertion of n = 1 is a classical result. Here we prove the case of 
n > 1. By the ltd formula (see QI3 Theorem 3.1]), we have 

d\\y(t)\\ 2 H = (2 (»(t),^i/(t) + o(t)), + + &(*)!&) dt + 2 (i/(t), S W (t) + 6(t)) dW t . 

Then we use the classical Ito formula to evaluate ||y(i)||jjj = (llf WlUr)"- From As- 
sumptions 12.11 and 12. 3[ we have 

\\By(t)f H <C(K)\\y(t)\\ 2 v , 
(y(t),By(t)) <K\\y{t)f H . 

Hence we deduce 

d\\y(t)\\ 2 H 

= n ||»(<)|^ n-1) [2 (y(t), Ay(t) + (i)>, + \\By(t) + b(t)f H ] dt 

+ 2n(n - 1) \\y(t)\\% n ' 2) Mi), By{t) + b(t))\ 2 dt 
+ 2n \\y{t)\\% n - 1) (y(t),By(t) + b(t)) dW t 

< n ||tf(*)|$ n-1) [2 (y(t),Ay(t)) t + \\By(t)\\% + 2 (By(t), b(t)) + 2 (y(t), a(t))] dt 
+ An(n - 1) \\y(t)\\ 2 ^- 2) \(y(t), By(t))\ 2 dt + n(4n - 3) \\y{t)\\ 2 ^ 1] \\b(t)\\ 2 H dt 
+ 2n \\y(t)\\ 2 ^- 1} (y(t),By(t) + 6(t)> dW t 

< n ||y(t)l^ n_1) [-f l|y(*)llv + C(k, K,n) \\b(t)\\ 2 H + 2 \\y(t)\\ H \\a(t)\\ H ] dt 
+ n{An - 3)K \\y(t)\\f dt + 2n ||l/(t)||^ n-1) (</(«), + &(«)) dW t . 



< 



2 IIi/WIIh" 1 K*)IIh + n) \\y(t)\\% 1 - 2 \\b(t)\\l d/ 

2 ™ ^+ _i_ lu.r+Mi 2 ^- 1 ) 



+ C(tf, n) ||i/(t)||2* dt + 2n \\y(t)\\T~ ' (y(t),By(t) + b{t)) dW t . (3.2) 
Define stopping time 

r k := inf ft E [0, 1) : E \\y(t)\\% > k\ A 1. 
Notice that from Young's equality, 

e / \\y(tAT k )\\x ll (* Al "*)L d * 

Jo 



<( sup E||y(tAT fe )|| 2 ;)^ A 1 (E||a(i)||^)^ d£ 



<7 sup E||y(t AT k )\\ 2 " + C(n,K,n)a 2n , 

4 t€[0,l] 



and similarly, 



f 1 1 

E / \\y(tAT k )\\ 2 "~ 2 \\b(tAT k )\\ dt< =r sup E\\y(tAT k )\\ 2 x+C(K,K,n)p n . 
Jo " ' 4 te[o,i] 

Then integrating (|3 . 2() on [0,t A r fc ), taking expectation, applying the Gronwall in- 
equality, and by the above two inequalities, we have 

sup E \\y(t A r fe )|| 2 ff n < C(k, K, n) (\\y Q f£ + a 2n + (3 n ) . 
*e[o,i] v 7 

Letting fc tend to infinity, we conclude the lemma. □ 

Remark 3.2. (1) The quasi-skew-symmetric condition is unnecessary in the case 
of n = 1, since the term 2n(n— l)\\y(t)\\% n 2 ^\(y{t), P>y(t) + b(t))\ 2 dt does not appear 
in this case. 

(2) This lemma can be extended to the case of any real number n G [1, oo), while 
some related techniques concerning n G (1,2) can be borrowed from e.g. [16]. 

Proceeding a similar argument, we have the L p -estimate for SEE (jl.ljl . 

Corollary 3.3. Under Assumptions Iff.ihEOl the solution x(-) to SEE (|l.ip 
satisfies 

sup E||a:(t)||£<a(K,JO(l+ sup E| u (t)|^ n ) 
se[o,i] V te[o.i] J 

with n = 1,2. 

4. Investigation into a stochastic bilinear functional. The purpose of this 
section is to prove Theorem l2.6l and study the properties of the operator- valued process 
P.. 

4.1. Proof of Theorem 12.61 For convenience, we write T T (A, B, M, N) as T T 
in the rest of this section, and hereafter denote C — C(n, K). 
Step 1. The well-posedness of T.. 

From Lemma [3~T1 equation (|2.3p has a unique solution for each £ g L 4 (Q, J>, H); 
moreover, replacing the expectation by conditional expectation in the argument, one 
can easily get 

sup E^\\z^(t)\\ 4 H <CU\\%- (4.1) 

t£[r,l] 

Thus T T (£,C) is uniquely determined (a.s.) for any £, £ 6 L 4 (fl, J>, H), and the 
estimate (|2.4I) clearly follows from the above inequality. 
Step 2. Construction of P. . 

Since (E^ Ai; r G [0, 1]) is a Doob's martingale and the filtration F is continuous, 
we can select and fix a continuous version of E^'Ai, denoted by A.. Now fix an 
arbitrary r € [0, 1]. Take a standard complete orthonormal basis {e^} of H . Then 
there is a set of full probability fli C £1 such that for each oj 6 fii, 

|[T T (e isei )] M| < Cv/aTR, Vi, j G Z+. 

Hence, from the Riesz representation theorem, there is a unique P T (w) G 23 (iJ) for 
each w £ Hi such that 



(a: 1 P T (w)i/) = [r T (a;,i/)](fa;), Vx,yGi/, 

7 



and 

||-Pt(w)||!B(H) < Cy/Kiw), Vw € fix. 

On the other hand, it follows from the definition of T T that (i) for each ui G Hi 
and any E G T T , 

[T T (xl E , yls)} (w) = [T r (x, y)] (uj), Vx, y G H; 

(ii) for each u; G fii and any Ei,E 2 G JV with EiH E 2 = 0, 

[T T (a;l Bl ,yl B2 )] (w) = 0, Vx,y e H. 

Thus, for any simple if- valued J>-measurable random variables £, we have 

(£(w),P T (a;)C(w)) = [T T (£ C)] («), g 



This along with a standard argument of approximation yields the relation 
Step 3. Weak stochastic continuity of P. . 

Fix an arbitrary r G [0, 1). Without loss of generality, we assume t < s. Then for 
any £, £ G £ 4 (fi, J T : we have 

E(£,(P s -P t )0 = E(^(l),M2 a 'C(l))+E/ 1 <z"'f(r),JV(r)2 a >C(r))dr 

«/ S 

- E <z*'«(l), Mz*' c (l)) -Si J (z* ,S (r), N(r)z^(r)) dr 
-FiJ <V'^(r), JV(r)z*< c (r)) dr. 
First, it follows from (|4. 1[) and Young's inequality that 

< CvJ* \\N(r)\\l {H) dr ■ ^E||£|& • E ||C||* -> 0, as |« - i| 0. (4.2) 

On the other hand, it is well-known that the trajectory of z T ^(-) is continuous in H 
(see [ID]), which along with (|4.1|) and the Lebesgue dominated convergence theorem 
implies 

E [ (z s ^(r),N(r)z sX (r))dr -E [ (/' S (r), N{r)z^ C (r)) dr 



< 



c (EAi) (y / E^y^iic^«(^+ v / ElCv / E^ r ^C ) 

-> 0, as |s — t| -> 0. 
Similarly, 

|E<z s '«(l),Mz s ^(l)) - E(z t -«(l),Mz t,< (l)>| ->0, as |a-t|->0. 
Therefore, we have for any £, £ G £ 4 (^, H)i 

limE(£,(P s -P t )C) =0, with s,ie [r,l]. 

S— 

The uniqueness of P. is obvious. Therefore, The proof of Theorem l2.6l is complete 



4.2. A property of P.. For each time re [0, 1), define 

B(fl,F T ,V) := {£ e L(n,J" T ,F) : ||£(-)||v is bounded}. 

Obviously, B(fl, IF T ,V) is dense in L i (Q,,F T ,H). 

Give the same four-tuple (A, B, M, N) as in Theorem 12.61 Let e g (0, 1 — r). For 
each £ G L 4 (il, H), consider the following equation 

zlHt) = J Az^(s)ds + j\Bz^( s )(t;T,0+s-h [T>T+e) t}dWs (4.3) 
and the following bilinear functional on L 4 (0, T T , if) with parameter e 

[T r £ (A,i?,Af,A0](£,C) :=E(z^(l),M^(l))+E J (z^{t),N{t)z^(t)) dt. 
From Lemma T3.ll equation (|4.3p has a unique solution such that 

sup V\\z^(t)\\ 4 H <CEM\\ 4 H - 

te[o,i] 

Thus T r E is well-defined. 

Next, we shall prove the following result concerning the relation between and 
P T , which plays a crucial role in the proof of the maximum principle. 

Proposition 4.1. Under the above setting, we have for each r G [0, 1) and any 
tC^L\n,F T ,H), 

E (£, P T C) = lim[T T £ (A, B, M, 2V)](fc C)- 

ej.0 



Remark 4.2. Compared with equation (|2 .3[) given the value £ in initial time t, 
the unknown process in equation (|4.3[) is affected by £ during a small period. The 
latter form appears in our derivation of the stochastic maximum principle. 

This proposition follows from several lemmas. For the sake of convenience, we 
write T^(A, B, M, N) as in this subsection. The hrst lemma gives a simple ap- 
proximation of z|T (t + e). 

Lemma 4.3. For each r g [0, 1) and each £ £ B{Q,,F T , V), we have 



E\\z^(t + e)- e"' (W T+E - W T ) if H < Ce ■ E . 
Proof. For the sake of convenience, we set 



Zl 



Then for te [r, r + e] , 

d(z t -&) = [A{z t -(,t) + e-^{W t -W r )A(\dt 

+ [B(zt-tt)+e~t(Wt-W r )Bt] dW t . 
By the Ito formula and some standard arguments, we have 



E \\ Z T + 6 - Ct+eHh < C 



< c 



e e-yE(||e||* |Wi-W T 
£ +S e-^EMWv^VlWt-Wrfdt 



= Ce 2 -E\\£f v . 
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(4.4) 



This concludes the lemma. □ 

Notice the fact that for any £, £ G B(Vl,F T1 V), 



T*(£, = E | T+£ N(t)z?<(t)) dt + E (r + e), P t+£ zJ-<(t + e)) 

Now we let e tend to 0. On the one hand, one can show that the term I\ tends to 
similarly as in (|4.2[) ; on the other hand, by means of the above lemma, the term I2 
must tend to the same limit as E (£ T +e, Pr+eCr+e) , where £t := e~~* (W T +t — Wt)£ 
and Ct := (W T+t - W T ) (. Therefore, we have 

Lemma 4.4. For each r e [0, 1) and any £, ( £ B(Q, T r ,V), we have 



lim 



Ele-ilWr+e-WrffoPr+eQ] ~T^,C) =0. 

Proof. Noticing that 

E^ 1 \W T+£ - W T \ 2 (£, Pr+eC) ] = E (£ r+£ , Pr+eCr+e) , 

we need only show 

Km |E(C T+£ ,P T+£ Cr+e) - J 3 | = 0. 

Indeed, from (|2.5p and Lemma T4.31 we have 

E|(£ r+£ ,P T+£ Cr +£ ) - (z^(r + e),P T+£ z^(r + e))\ 

<cVvM(Em%)Hn^HT+e)-( T+£ \\%y 

+ CVEAT(E||C|| 4 h) k (E||<'«(r + e) - &-f*||lr) 1 
— > 0, as e I 0. 

This concludes the lemma. □ 

On the other hand, from the continuity of P. we have the following 
Lemma 4.5. For each r € [0,1) and any£,( € B(Q,,jF t ,V), we have 



limE 

eJ.0 



E- l \W T+ e-Wr\ 2 {^{P r+s -P r )0 



= 0. 



Proof. It follows from (|2.5jl . the boundedness of £, £, and Doob's martingale 
inequality (see [5]) that 

\(^(P T+£ -P T )0\ 2 <C( K ,K)( sup E^'Ax) HellHllCllLei 1 ^)- 

v te[o,i] ' 

Then from Theorem 12.6( c) and the Lebesgue dominated convergence theorem, we 
have 

\E\S- 1 \W T+E - W T \ 2 (£, {Pr+e - P T ) ] f 
< E[e~ 2 \W T+£ - W T f]E |<£, {P T+ e -Pr)C)\ 2 

<3E|(e,(P T +e- J Pr)C>| 2 ^0, as e|0. 
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The lemma is proved. □ 

Finally we arrive at the position of completing the proof of Proposition 14.11 
Proof. [Proof of Proposition I4.1| Combining Lemmas 14.41 and 14.51 we have for 

each t G [0, 1) and any £ b , ( b G £(fi, T T , V), 

E(e,P T ( b )=liraTz(e,( b )- (4.5) 

Next, fix any £, ( G L 4 (0, J>, iJ). For arbitrary (5 > 0, we can find £ b ,C & G 
B(fi,.F,-, V) such that 

E|ie-e b ii 4 H + E|ic-c b ii 4 H <5 4 . 

Then one can show 



|E(£,P T C> -E(e b ,P r b C)| + |T T ^,C)-T r ^ b ,C b )| < C<5\/EAi [Edl^Hly + HCII/j )] ' 
Thus we have 

|E (£, P T - T*{t, 0| < |E <£ b , P r £ b ) - T^(e, e)\ + C6V^Ai[E(U\\ 4 h + UWh)] ' 
From (|4.5|1 and the arbitrariness of 8, we conclude the proposition. □ 



5. Proof of the stochastic maximum principle. In this section, we are going 
to prove our main theorem, the stochastic maximum principle. 

5.1. Spike variation and second-order expansion. Assume x is the state 
process with respect to a optimal control u. 

Following a classical technique in the optimal control, we construct a perturbed 
admissible control in the following way (named spike variation) 



it, if t G [t, t + e] 
u(t), otherwise, 



with fixed r G [0,1), sufficiently small positive e, and an arbitrary [/-valued J- T - 
measurable random variable u satisfying E| it I 4 / < oo. 

Let x e be the state process with respect to control u e . For the sake of convenience, 
we denote for tp = /, g,l, f x , g 

xi^xi fxx -i Qxx 1 ^XX ) 

<p(t) := tp(t,x(t),u(t)), 
p A (t) := <p(t, x{t), u e {t)) - <p(t, x{t), u(t)), 

<p% x {t) := / ip xx (t, x(t) + X[x e (t) - x(t)],u £ (t)) dX. 



From the basic L p estimates, we have 

Lemma 5.1. Under Assumvtions \2. 1W2.!A we have 

sup E||H(t)||^:= sup F,\\x e (t)-x(t)-xx(t)-x 2 (t)f H = o(s 2 ), 
te[o,i] te[o,i] 
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where x\ and x 2 are the solutions respectively to 

xi(t)= f [A{s) Xl (s) + f x (s) Xl (s) + f A (s)]ds 
Jo 



[B(s)x 1 (s)+g x (s)x 1 (s)+g A (s)]dW s 



x 2 (t) = I [A(s)x 2 (s) + f x (s)x 2 (s) + -f xx (s) (xi ® xi) (s) + f A (s)x 1 (s)} ds 



(5.1) 



+ / [B(s)x 2 (s) + g x (s)x 2 (s) + -g xx {s) {x x ® x x ) (s) + 5 ^(s)a;i(s)] dT^ s . 
Jo * 

(5.2) 

Proof. The proof is rather standard (see, e.g. [19]). so here we sketch the process. 
From Lemma I3JJ we have 



e" 2 supE||a;i(i)||^ +e" 1 supE||xi(0||^ +£" 2 supE|jx 2 (t)||^ < C, (5.3) 

t t t 

e- 2 su-p-E\\x £ (t) -x(t)\\ 4 H +e- 1 sup~E\\x £ {t) - x(t)f H < C, (5.4) 

t t 

£- 2 supE||a; e (t) - x{t) ~ x x {t)f H < C. (5.5) 



On the other hand, a direct calculation gives 



S(t) = / [A(s)E(s) + f x (s)E(s) + a £ (s)] ds 



+ / [B(s)E(s) +g x (s)E(s) + f3 £ (s)} dW s , 



where 



and 



« £ 00 :=f A (s)(x £ (s)-x(s)-x 1 (s)) 

+ g (/«»(*) _ fxx(s))(x £ (s) - x(s)) ® (x £ (s) - x(s)) 

+ \hx{s) ((x £ (s) - x{s)) <E> (x £ {s) - x(s)) - (xi <E> xi)(s)) , 



(3 £ (s) :=g A ( S )(x £ (s)-x(s)-x 1 (s)) 

+ ^(.9xx(s) - g X x{s)){x £ (s) - x(s)) <E) (x £ (s) - x(s)) 

+ ^9xx(s) {(x £ (s) - x(s)) ® (x £ (s) - x(s)) - (xt <g) xi)(s)) . 

From Lemma [3~T1 (j5.3[) - (|5.5[) and Legesbue's dominated convergence theorem we con- 
clude that 



sup E||S(i)|| 2 ff < 
te[o,i] 

The lemma is proved. □ 



E\\a £ (s)\\ 2 H y ds 



+ / V\\f3 £ (s)r H ds = o(e'). 
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With the aid of the above lemma and by the fact 

¥('))-^('))>0, 



we can prove the following result. 
Lemma 5.2. Under Assumptions 



have 



o(e) < E 



l A (t) + (L(t),x 1 (t) + x 2 (t)) + - (xi(t),l xx (t)xi(t)) 
(h x {x{l)), Xl (l) + x 2 (l)) + J (^(l),/^^!))^!)) 



dt 



(5.6) 



Proof. The proof is also standard (see, e.g. [H]), we give a sketch here. A direct 
calculation shows that 



< J(u%-)) - J(«(.)) 



E 



l A (t) + (/„(*), asi(l) + a; 2 (l)> + r (zi(t) + 



dt 



{^(5(1)), ^(1) + .x 2 (l)) + - ^(l), fe JSS (x(l))a: 1 (l)) + 7 (e), 



where 



7 (e) := E(^(i(l)),S(l))+E / (l x (t), 3(t)> dt 



i- 

i. 



^(5(1)) (z e (l)-5(l)),x £ (l)-5(l) 



+ -E (5(1)) (x £ (l) - 5(1)), x%l) - 5(1) - iri(l)) 
+ ^E (h xx (5(1)) (x £ (l) - 5(1) - Xl (l)), Xl (l)) 



-E 



£*(*)- M*) (x £ (t)-5(t)),x £ (t)-5(t)\ dt 



+ -E/ <r ra (t)(a; £ (t)-5(t)),^(t)-5(t)-a ; i(t)) dt 
* Jo 

i r 1 

+ ^E / <? ra (t)(a; £ (t)-5(t)-x 1 (t)),a;i(t)) dt 



with 



h% x := / ^(5(l) + A[^(l)-5(l)])dA. 
Jo 



Consequently, by 
we can deduce 



which implies the lemma. □ 



5.51) and the Legesbue dominated convergence theorem 
m\ = o{e), 
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5.2. First-order duality analysis. We need some duality analysis in order to 
tend e to in inequality (|5.6[) and get the maximum condition (|2.10l) . Recall the 
Hamiltonian 

H(t, x, u,p, q) = l(t, x, u) + (p, f(t, x, u)) + (q, g(t, x, u)) , 

and BSEE ([2~9f . Under Assumptions ED and E2J it follows from Du-Meng Propos- 
tion 3.2] that equation (|2.9[) has a unique F-progressively measurable weak solution 
(p, q) such that 



sup E||p(t)|| 2 ff + E f \\q(t)\\ 2 H dt<C(K,K)(l+ sup E|«(t)| 
te[o,i] Jo V te[o,i] 



(5.7) 



Thus the assertion (i) of Theorem 12.71 holds true. Furthermore, from Lemma 15.21 we 
have 

Lemma 5.3. Under A ssumptions \2.1H2.3l we have 
o(l) < e^eJ [H(t,x(t),u,p{t),q(t))-U(t,x(t),u(t),p(t),q(t))}dt 



(xi (t) , H xx (t, x (t) ,n(t), P (t) , q(t)) Xl (t)} dt 



+ -e- 1 E(a: 1 (l),^ x (S(l))a; 1 (l)), 



(5.8) 



where (p, q) is the solution to equation (|2.9p . 

Proof. From the duality between the SEE and BSEE (or by the Ito formula), and 
by (|5.3p and (I5.7p . we have 



E / [(l x (t), Xl (t)+x 2 (t))] dt + E(h x (x(l)),xi(l) + x 2 (l)) 



E 



l . 



E 



P(*), / (*) + 5 /xx(<) (XI ® X!) (t) + f*{t) Xl {t) 



q(t),g A (t) + -g xx (t) ( Xl <8> n) (t) + g*{t)xi{t) 



dt 



dt 



o{e) + E jf + * [(p(t)J A (t)) + (q(t),g A (t))} dt 
1 Z" 1 

+ -E / [(p(t),/ as (t)(si®a:i)(t)> + <g(t),ff M! (t)(a; 1 ®a:i)(t))] dt, 



this along with Lemma \5 . 2 1 yields 



0(1) < £- X E / [H{t,x(t),u s (t),p(t),q(t))-H(t,x(t),u(t),p(t),q(t))}dt 



£ - x E / {xi{t),H xx {t,x{t),u{t),p{t),q{t)) Xl {t)) dt 



+ -£~ 1 E( 2 : 1 (l),/ lra (x(l)) a : 1 (l)). 

Recalling the definition of u e , we conclude the lemma. □ 
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From the Lebesgue differentiation theorem, the first-order expansion part, which 
is the first term on the right hand side of inequality (15.81) , tends to 

E[%(t,z(t),u,p(t),<7(t)) - U{T,x(T),u(T),p(T),q(T))] , a.e. r 

for each u when e tends to 0. By the arbitrariness of u and some standard techniques, 
this yields the first term of maximum condition (|2.10[) . 

5.3. Second-order duality analysis and completion of the proof. In this 
subsection, we deal with the second order expansion part which is the second and third 
terms on the right hand side of inequality (|5.8[) . and complete the proof of Theorem 
1271 

Recall the four-tuple (A, B 7 M, N) with 

A(t) := A(t) + f x (t), B(t) := B(t) + g x (t), 

M := h xx (x(l)), N(t) := H xx (t,x(t),u(t),p(t),q(t)). 

Bearing in mind Assumptions I2.1H2.3I and the estimate (|5.7[) , we can easily obtain 
that the four-tuple (A,B,M, N) is "appropriate", and then from Theorem 12.61 there 
is a unique *B(-ff)-valued process P. as the Riesz representation of T. (A, B, Af, N). 
Hence, the assertion (ii) of Theorem 12.71 is proved. 

Observe that the solution X \ to equation (|5.1|) can be decomposed as 

where x^> and x^ are the solutions to the equations 

dx W (t) = [A(t)x (1) (t) + s-i f A (t)]dt + B{t)x {1 \t) dW t , 
dx {2) (t) = A{t)x (2) (t) dt + [B(t)x (2) (t) + e-^g A (t)} dW t , 

^ (1) |[0,r] =Z (2) |[0,t] =0. 

It follows from the i p -estimates that 

e- 2 sup E||x (1) (i)|| 4 +e _1 sup E||x (1) (t)|| 2 < C, 
te[o,i] te[o,i] 

sup E||a;( 2 )(t)|| 4 + sup E\\x^ (t)\\ 2 < C, 
te[o,i] te[o,i] 

which implies 

£- x E / <J;n(i),iV"(i)a;i(t)^ dt + e _1 E/x 1 (l),Ma:i(l)^ 

= o(l)+Ey (x^(t),N(t)x^(t)j dt + E (x^ {1), Mx& (I)). (5.9) 

Now we introduce the following equation 

z e (t) = j A(s)z e (s)ds + J [B(s)z e (s)+e-h [TtT+£) g A (T)]dW s . 

Then we have 

Lemma 5.4. For a.e. r G [0, 1), 

lim sup E|M 2 )(t) - z £ (t)||t = 0. 
te[T,i] 
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Proof. By the Ito formula, we have for each r G [0, 1], 

sup E\\xW(t)-z E (t)\\* < C(k,K)- - f +6 E\\g A (t)~g A (T)\\ 4 H dt. 

te[r,l] "" £ Jt 

From the Lebesgue differentiation theorem, we have for each X G L 4 (£l, F\, H) 



lim 

ej.0 £ 



-f E ||.g A (i) - dt = E|| 3 A (Y)-X||^, for a.e. r G [0, 1) 

- Jt 



Since L 4 (tt, Ti, H) is separable, let X run through a countable density subset Q in 
L 4 (fi, .Fi, if), and denote 

£" := £jf := {r : the above relation does not hold for X}. 

Then we have meas(£') = 0. For arbitrary positive r/, take an X G Q such that 

E||.g A (r)-X||^<ry, 

then for each r G [0, 1)\E, 

limi r +£ E|| ff A (0- 5 A (r)||;dt 

<lim- / + E|| 3 A (<) - X\\ 4 H dt + 8E,\\g A (T) - xf H 

< 16E|| 5 A (r) -Xf H < I677. 

From the arbitrariness of 77, we conclude this lemma. □ 
From the the above lemma and (|5.9p . we have 



£ _1 E / ^i(t),7V(t)xi(t)^ dt + e- 1 E^i(l),Mxi(l)^ 

= o(l)+Ey (V(t),iV(t)z £ (t)) di + E(V(l),Mz £ (l)^, Vtg[0,1)\£. (5.10) 

Keeping in mind the above relation, and applying Proposition 14. 1[ we conclude for 
each t G [0, 1)\E, 

E( ff A (r),P rff A (r)) =hme- 1 |E^ (x^t), N(t) Xl (t)) dt + E (ari(l), M Xl (l)^ . 



This along with Lemma 15.31 yields for each u G U, 

< E[^(T,£(r),« J p(T) jg (r))-K(r J x(r),fi(r),p(r) J g(r))] 
+ iE( 3 A (r),P T .g A (r)), a.e. rG [0,1). 

Therefore, the desired maximum condition (|2.10p follows from a standard argument; 
see, for example, This completes the proof of the stochastic maximum principle. 

Remark 5.5. Usually, the Lebesgue differentiation theorem is used at the end of 
the derivation of maximum principle. However, we utilize this sharp result at an early 
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step of the second-order duality analysis (Lemma \5.J^ . The benefit of such a different 
treatment can be seen from relation (|5.10[) . Indeed, the dynamic process Xi(-) on the 
left hand side of (I5.10[) is affected by a time-variant function g A (-), while the process 
z e (-) on the other side is related simply to a random variable g A {r). Noting the use 
of Proposition \4-l\ this skill is a key-point of our approach. 
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